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Abstract. 

We present a new spectrum extraction technique which employs opti- 
mal weights for the spectral extraction, deblends the overlapping spectra, 
determines the precise sky background, and takes into account correla- 
tions between errors correctly for STIS slitless observations. We obtained 
roughly 250 optimally extracted spectra in a deep STIS field as well as self- 
confirming redshift measurements for these objects, including a galaxy at 
z = 6.68. In addition, we identified five isolated emission-line objects in 
the dispersed image that were not accounted for by objects detected in 
■ the direct image. Assuming that these are Lya emission line galaxies at 

high redshifts and adopting a simple (^-function galaxy luminosity func- 
tion for these objects, we find, based on a likelihood analysis, that Lya 
| emission line galaxies at (z) ~ 4.69 may contribute at most 15% of the 

star formation rate density measured in Lyman break galaxies at z ~ 4. 
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Introduction 



Observations of distant galaxies have advanced rapidly as a result of the Keck 
telescope, which because of its large collecting area is sensitive to faint objects. 
^ ' But despite extensive searches, only four galaxies of redshift z > 5 have as 

yet been identified spectroscopically using the Keck telescope (Dey et al. 1998; 
Hu, McMahon, & Cowie 1998; Weymann et al. 1998; Spinrad et al. 1998). It 
has proven extremely difficult to identify high-redshift galaxies solely based on 
ground-based spectroscopy, because (1) galaxies lack prominent narrow-band 
features at rest-frame ultraviolet wavelengths (which are redshifted to observed- 
frame optical or infrared wavelengths) and (2) background sky light is the dom- 
inant source of noise at near-infrared wavelengths. 

We have sought to identify distant galaxies in very deep slitless spectroscopy 
observations acquired by STIS on board HST, by combining a new spectrum ex- 
traction technique with photometric and spectroscopic analysis techniques. Our 
analysis was designed to identify redshifts of galaxies in the slitless data by means 
of broad-band photometric techniques and to test these redshifts by identifying 
narrow-band emission, absorption, and continuum features in the same spectra. 
As a result of this analysis, we obtained optimally extracted spectra and self- 
confirming redshift measurements for roughly 250 object, including a galaxy at 
z = 6.68, and 5 isolated emission-line objects. 
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2. Data 



The very deep observations, obtained by HST using STIS toward a region of sky 
flanking the Hubble Deep Field, consisted of pairs of images: a direct image taken 
using no filter and a dispersed image taken using the G750L grating. Additional 
observations consisted of only a direct image. The integration time of the direct 
images totaled 4.5 h over 82 exposures, and the integration time of the dispersed 
images totaled 13.5 h over 60 exposures. We summed the direct and dispersed 
images using conventional image processing techniques. The summed image 
covers a sky area of 51 x 51 arcsec 2 . The spatial resolution of the summed direct 
and dispersed images is FWHM « 0.08 arcsec, the la detection threshold of the 
summed direct 26.2 mag arcsec , and the la detection threshold of 

the summed dispersed image at A « 9800 A is 5.5 x 10~ 18 erg s -1 cm~ 2 A -1 
arcsec -1 . 

3. Optimal Extraction of Slitless Spectra 

The spectrum extraction of the dispersed image is made especially difficult be- 
cause the image of the field is covered by light of faint galaxies, which when 
dispersed overlaps in the spatial direction and is blurred in the spectral direc- 
tion. To spatially deblend and spectrally deconvolve the spectra, we used the 
summed direct image to determine not only the exact object locations but also 
the exact two-dimensional spatial profiles of the spectra on the summed dis- 
persed image. The spatial profiles of these objects are crucial because (1) they 
provide the "weights" needed to optimally extract the spectra, (2) they pro- 
vide the models needed to deblend the overlapping spectra and determine the 
background sky level, and (3) they provide the spectral templates needed to 
optimally deconvolve the spectral blurring of extended objects. 

First, we identified objects in the summed direct image, using the SExtrac- 
tor program of Bertin & Arnouts (1996). Roughly 250 objects were identified 
in the summed direct image. Next, we modeled each pixel of the summed 
dispersed image as a linear sum of contributions from (1) relevant portions of 
all overlapping neighboring objects and (2) background sky: 



where are spectral elements of the fcth object, is the object position along 
the spectral direction in the direct image, A is the constant offset between object 
position and the starting pixel of the spectrum along the spectral direction, / 
is the object profile measured in the direct image, and B is the model sky (for 
which we found that a fourth order polynomial is necessary and sufficient). 

We treated the problem as a x 2 minimization problem, where the data were 
the pixel values of the summed dispersed image, the model was a linear sum of 
appropriate elements of the spatial profiles, and the parameters of the model 
were values of the spectral pixels. The x 2 is written 
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where J-% j is the flux measurement and <7jj is the \-a error at pixel 

Finally, we minimized x 2 between the model and the data with respect to 
the model parameters and obtained estimates of all model parameters simultane- 
ously to form one-dimensional spectra. The model parameters included roughly 
250,000 "object" parameters (from roughly 1000 spectral pixels each of roughly 
250 objects) and four thousand more "sky" parameters. As a zeroth order ap- 
proximation, the spectral elements of individual objects are independent of each 
other, and so we can solve the minimum x 2 separately for individual columns. 
The x 2 f° r column i is 
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Now we only need to solve for approximately 250 spectral elements Sf and (L+l) 
sky paremeters af (here L = 4) that give the minimum xj each time and repeat 
the calculation for all columns. Errors of the model parameters were obtained 
by solving the Hessian matrix at the minimum x\- 

This new spectrum extraction method is superior to other extraction meth- 
ods in many ways. First, the method employs optimal weights for the spectral 
extraction. Second, the method deblends the spectra. Third, the method deter- 
mines the precise sky background. Because the dispersed image is covered by 
light of extremely faint galaxies over most of its area, the spatial templates are 
needed to find the rare "clear" regions of sky background between the object 
spectra. Finally, the method estimates errors correctly by taking into account 
the correlation between spectral elements of overlapping objects. 

As a result of the spectrum extraction technique, we obtain 250 optimally 
extracted spectra for redshift analysis. To determine redshifts of extremely 
faint galaxies in STIS slitless spectra, we first measured photometric redshifts 
by means of a redshift likelihood technique (Fernandez-Soto, Lanzetta, & Yahil 
1999) and verified these redshifts by identifying narrow-band emission, absorp- 
tion, and continuum features in the same spectra. The goal is to obtain self- 
confirming photometric redshift estimates. 



4. Noise Characteristics 

The new spectrum extraction technique adopts object profiles determined from 
the direct image as model templates to determine the exact sky and to optimally 
extract spectra from the dispersed image. The analysis involves complicated 
smoothing and deblending. It is therefore not unreasonable to expect that the 
noise characteristics may have been highly skewed in the extracted spectra and 
do not follow a normal distribution any more. To proceed with discussions on 
the significance of detections of any spectral features, it is indeed necessary for us 
to first understand the noise characteristics of the final products of the analysis. 

To examine the noise characteristics, we form a histogram of pixel values 
of about 15 rows randomly chosen from the residual image. The histogram is 
shown in Figure 1, in which we also plot the best-fit Gaussian distribution with 



3 




Data Number 



Figure 1. Histogram of noise in the residual image. The dashed curve 
indicates a best-fit Gaussian profile. 



the full width at half maximum of 0.004. The corresponding 1-a deviation is 
0.0017, which is slightly larger than the 1-a deviation measured in the dispersed 
image (0.0015). The fact that the noise in the residual image follows a Gaus- 
sian distribution with a slightly larger 1-a deviation has secured our subsequent 
reference to a normal noise distribution for any further statistical analysis. 



5. Properties of Isolated Emission Lines 

In addition to the 250 objects observed in the direct image, which included a 
galaxy at z = 6.68, we identified five isolated emission lines in the dispersed 
image that were not accounted for by objects detected in the direct image. The 
z = 6.68 galaxy is the most distant galaxy that has yet been spectroscopically 
identified. Detailed analysis has been presented elsewhere (Chen, Lanzetta, & 
Pascarelle 1999). Here we focus on the five emission-line objects. 

To objectively identify these isolated emission lines, we applied the SExtrac- 
tor program to the smoothed dispersed image, setting the detection threshold 
such that nothing was detected in the negative of the image. Next, we removed 
the lines that correspond to objects identified in the direct image. Consequently, 
five isolated emission lines remained in the dispersed image. 

The nature of these isolated emission lines is uncertain because of the am- 
biguities in the wavelength determination. It is impossible to determine the 
wavelengths and therefore to calibrate the fluxes for these lines without first 
knowing their positions on the sky. We show in Figure 2 that the 1-a flux 
threshold of an unresolved emission line in the dispersed image may vary with 
wavelength by as much as a factor of four across the entire spectral range. How- 
ever, given the 1-a single pixel detection threshold of the direct image, « 26.2 
mag arcsec -2 , we can work out the 3-cr lower limit to the observed equivalent 
width (EW) as a function of wavelength for these isolated emission lines. The 
dot-dashed curve in Figure 2 indicates the 3-a lower limit to the observed EW 
of an unresolved line detected at a 3-cr significance level in the dispersed image 
versus wavelength. The actual observed EW limit as a function of wavelength 
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Figure 2. The solid curve is the l-a flux threshold of an unresolved 
emission line in the dispersed image versus wavelength. The dot-dashed 
curve shows the 3-a lower limit to the observed equivalent width. 

for a particular line can be obtained by scaling the dot-dashed curve to the sig- 
nificance of the line detection. According to the EW limits shown in Figure 2, 
the observed five emission lines are most likely to be high-redshift Lya emission 
lines, rather than low-redshift [O II] or Ha lines. 

The sensitivity curve shown in Figure 2 also indicates that the STIS ob- 
servations are more sensitive to faint emission-line galaxies than existing deep 
narrowband surveys from the ground (e.g. Hu, Cowie, & McMahon 1998). If 
these lines are high-redshift Lya emission lines, the statistics drawn from this 
observation may place a strong constraint on the number density of high-redshift 
Lya emission line galaxies and the star formation rate density contributed by 
this population. However, the study has been made extremely difficult because 
of the ambiguities in the wavelength (and therefore redshift) determination. 

To solve this problem, we applied a likelihood analysis to the observed 
emission lines, assuming that all five emission lines are high-redshift Lya lines 
and that the number density of these lines is well represented by a simple 5- 
function luminosity function, 

<p(L/L*) = (j)J{L/L* - 1). (4) 

The likelihood analysis returns a best-fit characteristic luminosity L* = 0.4 x 
10 42 h~ 2 evgs s -1 . Given the fact that five lines were detected, we find the number 
density to be 0* = 0.04 /i 3 Mpc -3 . 

On the basis of the simple 5-function galaxy luminosity function determined 
for the observed Lya emission line objects, we estimated the statistical properties 
of the galaxy population. First, we calculated the mean redshift of the observed 
lines and found (z) = 4.69. Second, we estimated the star formation rate den- 
sity, using the relationship between Ha luminosity and star formation rate of 
Madau, Pozzetti, & Dickinson (1998) and assuming case B recombination, where 
Lya/Ha « 10 (Osterbrock 1989). It turns out that the star formation rate den- 
sity of Lya emission line galaxies is 0.01 h M0yr -1 Mpc~ 3 for qo = 0.5 (without 
correction for extinction). This is about 15% of the star formation rate density 
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Figure 3. The predicted surface density of Lya emission line galaxies 
at different detection thresholds. The curves, from bottom up, corre- 
spond to a threshold of 10, 5, and 3 xl0 _18 ergss~ 1 cm~ 2 , respectively. 

measured by Steidel et al. (1999) for the Lyman break galaxies at z > 4, sug- 
gesting that galaxies exhibiting the Lya emission feature represent only a small 
portion of the whole galaxy population at high redshifts. We therefore con- 
clude that high-redshift galaxies are more efficiently and objectively identified 
using broad-band photometric rather than narrow-band imaging/spectroscopic 
techniques. 

Finally, we calculated the surface density of Lya emission line galaxies 
expected in observations of a given sensitivity. Figure 3 shows that we expect 
to detect no more that five Lya emission line galaxies per arcmin 2 at z > 4 that 
are stronger than 1 x 10 _17 ergs s _1 cm -2 and no more than 10 Lya emission line 
galaxies per arcmin 2 at z > 5 that are stronger than 5 x 10 _18 ergs s _1 cm -2 . 
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